A Survey on Multimodal Medical Data Visualization

Multimodal data of the complex human anatomy contain a wealth of information. To visualize and explore such data, techniques for emphasizing important structures and controlling visibility are essential. Such fused overview visualizations guide physicians to suspicious regions to be analyzed in detail, e.g. with slice-based viewing. We give an overview of state of the art in multimodal medical data visualization techniques. Multimodal medical data consists of multiple scans of the same subject using various acquisition methods, often combining multiple complimentary types of information. Three-dimensional visualization techniques for multimodal medical data can be used in diagnosis, treatment planning, doctor-patient communication as well as interdisciplinary communication. Over the years, multiple techniques have been developed in order to cope with the various associated challenges and present the relevant information from multiple sources in an insightful way. We present an overview of these techniques and analyze the specific challenges that arise in multimodal data visualization and how recent works aimed to solve these, often using smart visibility techniques. We provide a taxonomy of these multimodal visualization applications based on the modalities used and the visualization techniques employed. Additionally, we identify unsolved problems as potential future research directions.